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A SOUND CONTROL INSTALLATION 



FIELD OF THE INVENTION 

The present invention relates to an installation to 
control by sounds electrical units. 

It is known to control by sounds, for example in home 
applications, electrical xinits such as lamps, plugs*. • Using 
sounds (for example hand clicks) to control electrical units is 
particularly convenient as it does not require the user to 
physically act on a switch or a remote control apparatus. 

However, providing a space such as a room with sound 
controllable units poses several problems. 

A first problem is to distinguish a so\md order fran 
an accidental noise (for example, a fall of an object on the 
floor) . 

Another problem appears when more than one user is 
present in the room. Then, a conventional sound controllable 
unit cannot identify a user from another one. 

Another problem i s that the convent ional equipment s 
are not adapted to allow controlling more than one unit in a 
same room. In particular, providing a single room with two sound 
controllable units needs coding the sound control messages. This 
is complex and increases the number of environmental noises, 
which may be considered as parasitic control orders. 
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The present invention aims at providing a sound 
control installation for controlling electrical units which 
overcomes the drawbacks of the known equipments. 

Another purpose of the present invention is to provide 
5 an installation which does not require the user to physically 
act on a control element. 

Another purpose of the present invention is to allow 
controlling, in a same room, several electrical vmits without 
needing to individualize the control sound. 
10 Another purpose of the invention is to distinguish 

sound orders coming from different users in a same room. 

To attain these purposes and others, the present 
invention provides a remote control device capable of communi- 
cating with electrical ixnits to be controlled by means of wired 
15 or wireless links, only the control device being controllable 
with soxmd by a user. 

According to the present invention a schematic 3D-view 
of a room with the respective locations of electrical xmits to 
be controlled is displayed on a screen and, in a predetermined 

2 0 perimeter or area of the room, a hand of a user is tracked with 

stereo cameras .and is used to displace a. cursor on the screen. A 
3D-microphone ancay is also provided in order to identify the 
origin of a sound. 

Alternatively, a third camera on the opposite side of 
25 the room is used to take pictures of the real xmits to be 
controlled and/or for updating in real time the pictures of the 
control screen. 

The tracking of a hand in the area covered by the 
cameras uses a conventional shape recognition system in video 

3 0 pictures. Further, using a hand of a user as a "mouse" for 

pointing a cursor of a computerized screen is also known. For 
example, the Sony VAIO PCG-CIXS system has a piece of software 
called "Smart Capture Application" and a camera called "Motion 
Eye" which together have the ability to capture the index finger 
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of the user and link its movement to the movement of the mouse 
arrow on the computer screen. 

According to a preferred embodiment of the present 
invention, the installation is turned on by a sound Which is 
5 sensed by the microphones- Such an embodiment allows, as it will 
be better understood later, to distinguish several hands which 
could be present in the area covered by the cameras- The 
selected hand will be the closest from the location at which the 
soimd has been detected. 
10 The system can also check the pointed unit on the 

control screen by annoxmcing through a loud-speaker an identi- 
fiant of the pointed pictogram when the cursor comes on that 
pictogram . 

BRIEF DESCRIPTION OF THE DRAWINGS 

15 These purposes, features and advantages of the inven- 

tion will become apparent from the following detailed descrip- 
tion of embodiments, given by way of illustration and not limi- 
tation with reference to the accompanying drawings. 

Figure 1 represents very schematically a room provided 
2 0 with an installation according to the present invention; and 

- Figure. 2 illustrates . a . control screen used. -in an 

installation for controlling the room of figure 1. 

For clarity, only the elements usef\il to the under- 
standing of the invention have been shown in the drawings and 

2 5 will be disclosed hereafter. Especially, the programming steps 

which have to be made in order to implement the installation 
according to the present invention will not be detailed as they 
will readily occur to those skilled in the art. Further, known 
equipment for determining the location of a hand considered as a 

3 0 cursor, used in the present invention for the control screen, 

will not be disclosed as known, 
DETAILED DESCRIPTION OF THE INVENTION 

Figure 1 represents a room 1 provided with an instal- 
lation for soxmd controlling electrical elements according to 
3 5 the invention. 
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The control installation of the invention comprises a 
control device 2, a control screen 3 and image and sound 
sensors. In the represented embodiment, two video cameras 4 and 
5 are disposed on the comer of the screen 3 and three micro- 
5 phones 6, 7 and 8 are also disposed around the screen. Each 
microphone and camera is linked to the control device 2 which 
controls the screen 3. 

The sensors of the same type (video or audio) are not 
located at a same position so as to be capable of localizing the 
10 sound and picture sources. 

The microphones can be located anywhere in room 1 and 
linked (wire or wireless) to device 2. The video cameras 4 and 5 
are oriented to watch an area A located in front of the screen 3 
so as to film the hand H of a user U who wants to control the 
15 electrical units using soxond. 

The electrical units are, for example, a ceiling light 
10, two bracket lamps 11 and 12, two wall sockets 13 and 14, and 
a switch 15. These electrical units are distributed in the room 
and are linked to the control device 2. Each electrical lonit 
2 0 comprises a radiofrequency receiver RIO, Rll, R12, R13, R14 and 

, R15 ^.communicating with the control device 2 to receive control 

orders. Alternatively, the electrical units to be controlled by 
the installation according to the invention can be wire 
connected to the control device 2, 
25 An important feature of the invention is that the 

electrical xrnits that will be rendered sound controllable by the 
present invention are not individually sound controllable* 

Screen 3 is not necessarily contiguous with the 
control device 2 provided that it is linked to this device and 
30 it is visible from the area A watched by cameras 4 and 5, and 
within line of sight from the microphones. For example, screen 3 
can be the screen of a TV set equipped to be controlled by 
device 2. Then, the area A covered by the cameras 4 and 5 is 
preferably the area from which the users might be watching TV, 
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for example an area around a sofa 20 disposed in front of the 
screen 3 . 

Figure 2 represents an image on screen 3 when the 
installation is on. According to the invention, the representa- 
5 tion is preferably a perspective to display not only the wall Wl 
of the screen but also the floor Fl, the roof ROl, and the walls 
WRl and WLl, respectively right and left to the wall Wl. 

On the screen, the control device 2 displays not only 
the shape of the room 1 but also, according to a preferred 
10 embodiment, pictograms PIO, Pll, P12, P13, P14 and P15 respec- 
tively figuring the imits 11, 12, 13, 14 and 15 to be controlled 
by the installation. 

The generation of pictograms P is obtained in a 
configuring phase of the software controlling the control device 
15 2. 

According to a first variant, the user (or the 
installer) defines the walls of the room and the locations of 
the pictograms using a conventional graphic software. 

According to a second variant, the installation auto- 

2 0 matically acquires the controllable units at each activation of 

. ... .the installation. According .to .that embodiment,., a. third„..camera.. 
(not shown) is provided to film the wall of the room containing 
the screen and to be able to locate all elements in the room. 
Manual registration could be included as the control space is 
25 3D, and hence easily definable through a coordinate representa- 
tion on a computer. The selection of the elements of the room 
which have to be displayed on the control screen 3 as pictograms 
is then made by using the communication links between the 
controllable lanits and the control device 2. For exaittple, the 

3 0 elements RIO to R15 are not only radiof requency receivers but 

also radiof requency emitters. Then, when the installation is 
turned on, a request message is sent to all the possibly 
connected electrical units. The respective units respond with an 
identifier to allow the identification of the different units. 
35 If necessary, the transmission between the various units and the 
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control device 2 is also used to assist the localization made by 
the video system. The identification of the various electrical 
units can be used to automatically select a pictogram (socket, 
switch, lamp, etc.) chosen in a library of the installation. 
5 According to a third variant, the representation of 

the control screen 3 is a real representation. Then, the trans- 
ceivers of the controllable units are only used to locate on the 
picture the area in which the cursor has to be considered as 
selecting a xmit. 

10 The operation of the installation is fqr example as 

follows. The cameras 4 and 5 permanently monitor the area A 
(figure 1) and the images are processed to identify the presence 
of a hand. Known detection systems of human shapes like hands 
usually use colour differentiation to more quickly isolate the 

15 skin area on a picture. The detection of the position of a hand 
in a dedicated area is made by conventional techniques. If 
needed, a reference object can be disposed in the field of the 
cameras to help in matching the referentials of the images of 
the cameras. 

2 0 Once the hand H of a user U (figure 1) has been 

detected,., the system calculates the displacements of. the hand 
between two successive video pictures to transfer this movement 
on the cursor C displayed on the control screen 3. Then, the 
user can see the cxarsor C move along with his hand displacement 

25 to select a unit to control. 

Preferably, once the cursor C encounters on the screen 
a pictogram P of a unit, the user is made aware that a unit can 
be selected. For example, the system can annoxmce through 
speakers the name and type of the unit selected. Alternatively, 

30 the corresponding pictogram can be highlighted on the screen. 

Having selected a controllable unit, the user can make 
a click with his hand, or produce another noise, to control the 
corresponding unit. This sound is sensed by the three micro- 
phones 6, 7 and 8 and processed to check that it originates from 

35 the hand H, or its close neighbourhood. For this purpose, the 
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control device 2 calculates the difference between the times of 
arrival of the sound on the different microphones • Knowing the 
location of the microphones, the system is then capable of 
calculating the 2D or 3D location of the soxind source. 
5 If the origin of the soxmd substantially corresponds 

to the location of the hand H detected by the cameras, then the 
installation executes the appropriate control. 

If not, the installation ignores the sound order which 
is to be considered as a parasitic noise. 
10 An advantage of the present invention is that the 

installation is able to distinguish perturbating noise from a 
control order. 

Another advantage of the system of the present inven- 
tion is that it is possible to individiaally control more than 
15 one unit is the same room. 

To distinguish the hands of a user and select only one 
hand for controlling the cursor C, various solutions can be 
adopted. A first solution is to consider the hand controlling 
the cursor as that with the highest degree of motion. Then, the 
2 0 user wanting to use the system knows that he has to move one 
hand in the area A. more .than the .other and that this Jtiand will, 
be used to control the cursor. Further, the user will see that 
the cursor displayed on screen 3 follows the displacement of his 
hand. Another solution is to identify the position (open or 

2 5 closed) of the hands and, preferably, the existence of a finger 

pointed in the alignment of the arm of the user and to select 
this shape for controlling the cursor on the screen. The imple- 
mentation of that solution only needs the control device 2 to be 
equipped to detect shapes like arms and hands. 

3 0 According to a preferred enibodiment of the present 

invention, the selection of the hand of a user which has to 
control the position of the cursor C is made at the activation 
of the installation, and the installation is sound activated. In 
other words, the three microphones 6, 7 and 8 permanently listen 
3 5 to the noise in the room, the cameras 4 and 5 being off. When 



detecting a sound which may be considered as an hand click, the 
control device 2 switches on cameras 4 and 5 and the first 
pictures taken by the cameras are analyzed in an area corre- 
sponding to the origin of the sound calculated by triangulation 
5 with microphones 6, 7 and 8. If a shape recognized as a hand is 
located in this area, the installation considers that this hand 
has to be tracked to control the cursor location on the screen. 
If no hand shape is detected in this area, the installation 
considers that the noise is a parasitic one* Alternatively, both 
10 cameras and microphones can monitor the room permanently. 

Also, one could also provide two or more cursors on 
the screen corresponding to more than one hand. Then, all the 
cursors will be followed by the camera and the selection of the 
cursor to be taken into account when a sound is produced can be 
15 made by controlling the origin of the sound. 

The practical implementation of the present invention 
is in the ability of one ordinary skilled in the art in view of 
the functional explanations above. In particular, the program- 
ming of a software to be used for initializing and operating the 
2 0 installation according to the invention is not to be detailed as 

^™it is- in the ability, of those, skilled in the .art . ^- , 

Even if the present invention has been disclosed in 
connection with a particular application to home environment, 
the invention more generally applies to any environment or space 

2 5 where a soiand control installation can be used for controlling 

more than one unit. For exairple, the area surveyed by the 
installation can be a car, a storage house. . . 

Various means can be used to determine the origin of 
the soxmd; for example, in some simple installations it can be 

3 0 sufficient to use two microphones. Also four microphones could 

be used. Three microphones only determine position of the hand 
in two, say x and y dimensions. Two cameras are able to deter- 
mine all three dimensions, the x and y axes being given by both 
cameras, and the z-axis is deriving from the disparity map 
3 5 computed by combining simultaneous pictures acquired by the 
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camera pair. With three microphones, redundancy is obtained 
between x and y axes across sound and pictures, which enables 
the system to check that the hand is where the approved sound is 
coming from. A fourth microphone mounted at a wall of the room 
5 (not the same as the display) would enable the cross-check of 
valid sound command of hand position across all three dimen- 
sions • 

Having thus described at least one illustrative 
embodiment of the invention, various alterations modifications 

10 and iitprovements will readily occur to those skilled in the art. 
Such alteration, modification, and iitprovements are intended to 
be within the spirit and scope of the invention. Accordingly, 
the foregoing description is by way of example only and is not 
intended to be limiting. The invention is limited only as 

15 defined in the following claims and the equivalent thereto. 
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CLAIMS 

1. A sound control installation for at least one elec- 
trical tmit comprising: 

at least two cameras (4, 5) to take pictures of a 
determined area (A) in a space containing the electrical xinits; 

at least two microphones (6, 7, 8) positioned at 
different locations to sense the sounds in said space; 

a control screen (3) displaying an image of the space 
and the electrical units; 

a control device for positioning on the control screen 
a cursor (C) in accordance with the movements of the hand of a 
user detected by said cameras, and for controlling a determined 
electrical unit when: 

- the cursor is on the image of said determined electrical 
unit, 

- a soixnd is produced, and 

- a system associated with the microphones checks that the 
origin of the sound is close to the position of the hand. 

2. The installation of claim 1, in which said at least 
one electrical \jnit communicates with the control device (2) 
through wired link(s) • 

3. The installation of claim 1, in which said at least 
one electrical imit communicates with the control device (2) 
through wireless lihk(s) * 

4. The installation of claim 3, in which the wireless 
link(s) use radiof requency transceiver (s) . 

5. The installation of claim 1, in which each electri- 
cal xmit is identified on said control screen (3) by a pictogram 
located in a picture representing said space. 

6. The installation of claim 1, in which several 
cursors (C) are displayed on said control screen (3) , each 
cxirsor (C) following the displacements of a hand (H) in the 
surveyed area (A) of the cameras (4, 5) . 

7. The installation of claim 1 further comprising a 
third camera to film a picture representing said space and the 
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electrical imit(s) to be controlled, the third camera being 
located in order to film the room from a location not being 
comprised between said determined area (A) and the control 
screen (3) . 

8, A method for controlling the installation according 
to claim 1, in which the installation is turned on further to 
the detection of a sound in said space. 

9, The method of claim 8, in which the hand control- 
ling the cursor (C) on the control screen (3) is chosen by 
matching the detected origin of the activation soiind and the 
location of the hand detected by the cameras* 

10. The method of claim 1, in which, when the cursor 
comes on the pictogram of an electrical unit on the control 
screen (3), the corresponding pictogram is lighted. 

11. The method of claim 1, in which, when the cursor 
comes on the pictogram of an electrical unit on the control 
screen (3), the corresponding electrical unit is identified by a 
sound message « 
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Abstract 

The invention concerns a soimd control installation 
for at least one electrical unit coitprising two cameras (4, 5) 
to take pictures of an area (A) in a space containing the elec- 
trical units; three microphones (6, 7, 8) positioned at differ- 
ent locations to sense the sounds in said space; a control 
screen (3) displaying an image of the space and the electrical 
units; a control device for positioning on the control screen a 
cursor (C) in accordance with the movements of the hand of a 
user detected by the cameras, and for controlling an electrical 
vinit when: the cursor is on the image of an electrical lanit, a 
sound is produced, and -a system associated with the microphones 
checks that the origin of the sovind is close to the position of 
the hand. 
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